Skip to content

feat(ccproxy): v2.0.0 — inspector architecture, lightllm, DAG pipeline, compliance#16

Open
starbaser wants to merge 385 commits into
mainfrom
dev
Open

feat(ccproxy): v2.0.0 — inspector architecture, lightllm, DAG pipeline, compliance#16
starbaser wants to merge 385 commits into
mainfrom
dev

Conversation

@starbaser

@starbaser starbaser commented Apr 16, 2026

Copy link
Copy Markdown
Owner

AI Summary

Complete rewrite of ccproxy from a LiteLLM proxy subprocess model to an in-process mitmproxy-based transparent LLM API interceptor. This is the v2.0.0 release (tagged v2.0.0-rc1).

  • Inspector architecture: mitmweb runs in-process via WebMaster API with dual listeners — reverse proxy + WireGuard namespace jail. No subprocess, no gateway server.
  • lightllm: Surgical nerve connector into LiteLLM's BaseConfig transformation pipeline, bypassing cost tracking and callback machinery entirely.
  • DAG-based hook pipeline: @hook(reads=..., writes=...) decorator-declared data dependencies, topologically sorted via Kahn's algorithm. Per-request overrides via x-ccproxy-hooks header.
  • SSE streaming: SseTransformer stateful stream callable — parses, transforms per-chunk via LiteLLM's provider iterators, re-serializes as OpenAI-format SSE.
  • Compliance profile learning: Provider-agnostic system that observes legitimate request shapes from WireGuard traffic and stamps compliance profiles onto proxied requests.
  • Gemini/Vertex AI support: Full routing, OAuth handling, context caching via cachedContents API, path rewriting for cloudcode-pa.googleapis.com.
  • Flows CLI: ccproxy flows list/dump/diff/compare/clear with multi-page HAR 1.2 output, jq filtering, and sliding-window diff across flow sets.
  • MCP notification endpoint: POST /mcp/notify for terminal event ingestion, buffered and injected as synthetic tool_use/tool_result pairs.
  • XDG config directory: Default config moved to ~/.config/ccproxy/ (breaking change).
  • init replaces install: CLI rename (breaking change).
  • Rich pipeline visualization: render_pipeline() builds a full DAG display with parallel groups via rich.columns.Columns.

Breaking Changes

  • Config directory: ~/.ccproxy/~/.config/ccproxy/
  • CLI: ccproxy installccproxy init
  • --debug flag replaced by --log-level / -v
  • forward_port / reverse_port replaced by unified port config
  • mitm config section renamed to inspect
  • Prisma/database infrastructure removed entirely
  • LiteLLM proxy subprocess removed
  • to_mermaid / to_ascii removed from HookDAG

Test plan

  • just test passes with ≥90% coverage
  • just lint / just typecheck clean
  • Smoke test: ccproxy run --inspect -- claude --model haiku -p "what's 2+2"
  • Verify ccproxy init creates config at ~/.config/ccproxy/
  • Verify flows CLI: ccproxy flows list, ccproxy flows dump
  • Verify Gemini routing through inspector

extract_session_id declared writes=["session_id"] but now writes to
flow.metadata — update to writes=[]. inject_mcp_notifications read
session_id from ctx.metadata (body) which was always empty after the
previous fix — read from flow.metadata instead.
Hardcoded 40-char width caused right border misalignment when parallel
group labels overflowed. Width now computed from longest content line.
…urce

The LiteLLM proxy server was removed several commits ago but many files
still described the old architecture. This commit systematically removes
every stale reference: rewrites README, configuration, and inspect docs
from scratch; deletes the superseded skills/using-litellm-ccproxy skill;
drops 8 unused dependencies from pyproject.toml; removes 9 dead type
stubs; fixes source docstrings/comments/types across 6 source files;
and cleans infrastructure files (process-compose, docker-compose, nix
module, .gitignore).
# Conflicts:
#	README.md
#	pyproject.toml
#	src/ccproxy/templates/config.yaml
# Conflicts:
#	pyproject.toml
Remove stale litellm-db postgres reference from Docker services,
correct type stubs listing (litellm/opentelemetry/xepor, not mitmproxy).
…d retry

Remove dead oauth_ttl/oauth_refresh_buffer machinery — tokens were loaded
once at startup and never proactively refreshed. Now on 401, the credential
source is re-read (file) or re-run (command); if the token changed, the
request is retried with the fresh value via httpx. Unchanged tokens fail
through as truly stale credentials.

Also moves skills/ to plugin root per Claude Code plugin spec and updates
plugin.json to reflect the current mitmproxy-based architecture.
…tion

Extract shared CredentialSource base model from OAuthSource — supports
`file` (read path) and `command` (run shell) credential resolution.
MitmproxyOptions.web_password now accepts CredentialSource for stable,
deterministic mitmweb auth via 1Password or opnix secrets.

Fix mitmproxy web_password: update_defer doesn't trigger WebAuth.configure,
so web_password is now set via opts.update() after WebMaster creation.
Status command resolves the credential source to show the full tokenized URL.
Capture the full pre-pipeline client request (method, URL, headers, body)
in InspectorAddon.request() before any hooks mutate the flow. Expose it
via a custom mitmproxy content view at /flows/{id}/request/content/client-request
and a ccproxy.clientrequest command for structured JSON access.

Renames OriginalRequest → ClientRequest using canonical MITM terminology:
client request (what the caller sent) vs forwarded request (post-pipeline).
…ystem

Replace hardcoded add_beta_headers and inject_claude_code_identity hooks
with a dynamic observation-based system that learns compliance contracts
from legitimate CLI traffic and applies them to SDK requests.

Observation is built into InspectorAddon.request() pre-pipeline, reading
raw ClientRequest snapshots from WireGuard flows. Application runs as
the last outbound pipeline hook on reverse proxy flows after transform.
Profiles are persisted to {config_dir}/compliance_profiles.json and
keyed by (provider, user_agent). An Anthropic v0 seed profile bootstraps
from existing constants to prevent regression.
… defaults

Add x-goog-api-key to HEADER_EXCLUSIONS (Google's API key header should
not be stamped onto other requests). Update nix/defaults.nix to use the
new compliance hooks instead of the deprecated add_beta_headers and
inject_claude_code_identity.
Replace deprecated add_beta_headers/inject_claude_code_identity hook
references with apply_compliance. Document the compliance/ subsystem
and add ProfileStore to singleton patterns.
Three transform modes: redirect (default) preserves request body and
rewrites destination host for same-format flows (Anthropic→Anthropic,
Gemini→Gemini). Transform mode runs lightllm for cross-format
conversion. Passthrough leaves everything unchanged.

Also adds dest_host field to TransformRoute and excludes x-goog-api-key
from compliance profiling.
WireGuard flows already have the correct destination — transform and
redirect rules should only apply to reverse proxy flows. WireGuard
flows that match a non-passthrough rule now fall through to passthrough
instead of having their auth tokens overwritten.
…le lookup

Fix extractor to lowercase header names before UA lookup (ClientRequest
preserves original case, so "User-Agent" != "user-agent"). Rename
get_best_profile to get_profile with ua_hint parameter for targeted
lookup. apply_compliance reads OAuthSource.user_agent as the hint to
select the correct profile for the provider.
The forward_oauth hook was writing ccproxy_oauth_provider into the
request body's metadata dict, which Anthropic rejects as unknown field.
Move to flow.metadata["ccproxy.oauth_provider"] — same pattern as
session_id. Update the 401 retry reader in addon.py to match.
…om merge

These are feature config fields, not compliance requirements. Stamping
them onto SDK requests forces extended thinking and max effort on every
request. Headers, system prompt, and metadata remain as compliance
essentials.

Also adds Gemini transform rule to dev config and fixes duplicate
user_agent in oat_sources.
Detect when observed API requests nest the payload inside a wrapper
field (e.g. cloudcode-pa's {project, user_prompt_id, request: {body}}).
Store body_wrapper on profile, auto-wrap SDK requests at merge time.
Generate user_prompt_id fresh per-request (like session_id).

Gemini CLI through ccproxy now works: standard Gemini body → compliance
wraps it → redirects to cloudcode-pa.googleapis.com → 200 OK.
- Add dest_path to TransformRoute for path rewriting in redirect mode
- Extract model from URL path (/models/{model}:method) for body wrapper
- Support x-goog-api-key sentinel detection in forward_oauth
- Clear x-goog-api-key after token injection

Gemini SDK requests through ccproxy now work end-to-end:
/gemini/v1beta/models/X:generateContent → compliance wraps body →
redirects to cloudcode-pa.googleapis.com/v1internal:streamGenerateContent
…uth fixes

Add `ccproxy flows` subcommand for querying mitmweb flows API (list, req,
res, client, diff, clear). Built on ccproxy config infrastructure with
httpx and rich output.

Fix forward_oauth: remove UA override from _inject_token() so compliance
profile handles it, replace unconditional x-goog-api-key clear with
conditional sentinel loop to prevent double-clear when auth_header targets
that header.

Add Gemini SDK path rewriting in redirect handler: strip routing prefix
(/gemini/) and map standard genai SDK paths to cloudcode-pa's /v1internal
endpoint. Add cloudcode-pa response envelope unwrapping in InspectorAddon
so the genai SDK receives standard Gemini format.

Update nix/defaults.nix with transforms, compliance config, and gemini
oat_sources destinations.
… skill

Add two-part architectural doc (docs/inspector-and-compliance.md) covering
the inspector MITM system and compliance learning system.

Add using-ccproxy-inspector skill with Python helper scripts:
- list_flows.py: enriched flow listing with provider/model/status filters
- inspect_flow.py: client-vs-forwarded request diff with change summary
- compliance_status.py: profile/accumulator status from on-disk store

Update using-ccproxy-api skill to reflect current defaults: compliance-based
headers/identity instead of explicit add_beta_headers/inject_claude_code_identity
hooks, redirect mode in transform routes, 401-triggered token refresh.
…plate

Rewrite SKILL.md with proper installation guide covering Home Manager
module, standalone setup, and per-project mkConfig instances. Add
configuration reference section with full ccproxy.yaml example.

Rewrite troubleshooting.md to match current inspector architecture:
remove stale references to rule_evaluator, model_router, --detach,
--mitm, TTL-based refresh, flat hooks list. Replace with compliance-
based diagnostics, flow inspection commands, and correct hook pipeline.

Update template ccproxy.yaml to match nix/defaults.nix: outbound hooks
now use inject_mcp_notifications, verbose_mode, apply_compliance
instead of add_beta_headers and inject_claude_code_identity. Add
compliance and gemini oat_sources sections.
The directory-exists guard blocked `ccproxy install` in any pre-existing
config dir (e.g. ~/.ccproxy created by a previous start) even when the
yaml file wasn't there yet. Now only checks per-file existence — creates
the directory silently and skips individual files that already exist
unless --force is passed.
Introduces a `mode` field to distinguish between redirect and transform
operations. Updates `InspectorAddon.responseheaders` and transform route
handlers to check this mode before processing streaming transforms.
Remove debug artifacts (check_auth.py, .env.example), untrack .mcp.json,
fill LICENSE placeholder, fix GitHub URLs, sync dev dependency pins,
strip ~700 lines of redundant docstrings/comments across 52 files,
rewrite change-history comments, fix dead code (mid-file imports,
vestigial case variants, empty TYPE_CHECKING block).
Tools inside the namespace (e.g. PAL MCP server) configured with
localhost base URLs couldn't reach host services — 127.0.0.1 is the
namespace's own isolated loopback. This caused connection refused for
any tool hardcoded to http://127.0.0.1:4000.

- Enable route_localnet sysctl so iptables OUTPUT DNAT works on loopback
- Add OUTPUT DNAT rule: 127.0.0.1 → 10.0.2.2 (slirp4netns gateway)
- Add port remap rule when running port differs from default (4000→4001)
- Pass proxy_port from cli.py to create_namespace()
- Atomic write for combined CA bundle via tempfile+rename
…uting

- Document namespace localhost→host DNAT routing and network topology
- Add ccproxy flows CLI commands to CLI reference
- Document tools/flows.py MitmwebClient subsystem
- Add Gemini-through-inspector routing notes (PAL + Gemini CLI paths)
- Fix inspector UI port (8083→8084), note dual-instance dev/prod setup
- test_cli.py: update TestInstallConfig tests to match new install behavior
  (no SystemExit on existing files, "Installed" not "Copied", "Configuration
  installed to:" not "Installation complete!")
- test_response_transform.py: add mode="transform" to TransformMeta fixtures
  so cross-provider and non-streaming response tests actually exercise the
  transform code path instead of silently hitting the redirect default
starbaser added 11 commits May 23, 2026 11:00
…nder

Three independent ergonomic improvements landed together; zero behavior
change.

- Naming pass. ListenerFormat -> InboundFormat (StrEnum) so the type name
  matches the canonical inbound/outbound axis used everywhere else.
  Provider.provider -> Provider.type so the field matches the
  AuthSource.type discriminator pattern. TransformMeta.provider ->
  .provider_type, TransformMeta.listener_format -> .inbound_format.
  Dispatch kwarg renames: upstream_provider/provider -> provider_type,
  listener_format -> inbound_format. Metadata key ccproxy.listener_format
  -> ccproxy.inbound_format. _select_listener_format ->
  _select_inbound_format. Nix-side YAML: providers.X.provider ->
  providers.X.type in nix/defaults.nix + bundled template.

- Context.extras. ~60 LOC typed accessor (.get/.set/.delete/.has) over
  ctx._body via glom, exposed as layer 3 of the three-layer access model
  alongside the header and typed-IR layers. Existing glom(ctx._body, ...)
  callers stay valid; migration is opportunistic.

- HookDAG.render(). Emits stateDiagram-v2 mermaid markup walking the
  topo-sorted execution order with [*] brackets for sources/sinks.
  ccproxy status --mermaid prints inbound + outbound DAGs as paste-ready
  output.

AGENTS.md + docs/lightllm.md updated to reflect the renames, the new
Context.extras layer, and the --mermaid CLI flag. phase4.md added as the
next-session plan for OpenAI Responses (Codex parity).

Verified: 1671 tests pass, mypy clean across 103 source files, grep for
ListenerFormat / listener_format / upstream_provider / _listener_format
returns zero matches in src/ tests/ docs/ AGENTS.md nix/.
Apply Tier 1+2+3 cuts from the removal-candidates plan:

- Delete pure duplicates: Marketplace Plugin Sync, Defaults Flow
  diagram, MCP tool enumeration, transport constants, FlowRecord
  field listing, historical commit references.
- Compress subsystem deep-dives with canonical homes elsewhere:
  lightllm (docs/lightllm.md), Perplexity Pro narrative
  (docs/pplx.md), oauth/sources prose, Anthropic billing two-phase
  signing (regenerate.py docstring), inspector + pipeline per-file
  enumerations, dev-vs-prod section.
- Selective trim: hook table Purpose column to single-sentence form,
  Configuration narrative dedupe, Smoke Test prose, SSL/Logging
  Implementation Notes entries.

Preserve all load-bearing content: both IMPERATIVE blocks (shape
replay; Perplexity docs gate), Triage Principle, three-layer access
model, hook table rows, sentinel-key concept, routing precedence,
Key Constants, Body metadata footgun, SSE streaming + namespace
localhost routing notes.
Enables bidirectional transform for OpenAI's Responses API (used by
Codex CLI). Handles 27-item discriminated union in input[], preserving
reasoning blocks and server-side tool calls via raw_extras for lossless
round-trip.
Implements listener-side rendering for InboundFormat.OPENAI_RESPONSES,
enabling ccproxy to serve OpenAI Codex CLI traffic via the /v1/responses
streaming protocol with per-item and per-content-part lifecycle events.
Introduces an interactive REPL for flow inspection and ships sanitized
default shapes for Anthropic and Gemini providers. User-captured shapes
override bundled defaults. Patch-series support allows incremental shape
modifications via quilt-style unified diffs.
Consolidates shape storage under a single shapes_dir, with provider
patch queues living as {provider}/series subdirectories instead of a
separate patches_dir. Simplifies configuration and aligns the on-disk
layout with the quilt-style patch workflow.

BREAKING CHANGE: removed ShapingConfig.patches_dir; patch queues now
  live under shapes_dir/{provider}/
Shape-backed fingerprint profiles (e.g., 'anthropic') now resolve
through provider .mflow metadata instead of requiring hardcoded
curl-cffi browser names. FingerprintCaptureAddon parses native TLS
ClientHello into JA3/JA4 material, ShapeCaptureAddon embeds it in shape
metadata, and transport dispatch replays it via curl-cffi custom
options.
Updates all documentation to reflect the supported metadata access
pattern (ctx.metadata / metadata_from_flow) instead of the internal
mitmproxy backing store (flow.metadata). The API itself was already in
place; this aligns docs with actual usage.
…anitizer

- Strip x-ccproxy-flow-id at capture time (shape_capturer)
- New EgressSanitizerAddon: drop ccproxy-internal correlation headers
  before mitmproxy forwards (x-ccproxy-flow-id, -hooks, -oauth-injected)
- Add diagnostics to anthropic content_fields so capturer's
  previous_message_id never replays onto another user's request
- New scripts/package-mflows.py: one-way distillation of personal
  captures into bundled templates (strips identifier headers,
  zeroes metadata.user_id, drops body messages/tools, trims system
  to first 2 entries, keeps only ccproxy.fingerprint.profile in
  flow metadata)
- Pre-commit hook: --verify mode rejects bundled shapes that carry
  capturer identity
- Re-derive src/ccproxy/templates/shapes/anthropic.mflow from a
  fresh capture using the new minimal scrubber
- Delete tests/test_shaping_defaults.py (contained author PII in
  literal hand-curated marker list — replaced by structural verify
  step in package-mflows.py)
- Apply HTTP_CONTENT_DECODING=0 in transport/dispatch +
  CapturedFingerprint.transport_kwargs so curl-cffi stops
  auto-decompressing and mitmproxy decodes Content-Encoding itself
- Extract _default_hooks() factory in config.py (resolves ty
  diagnostic on Field default_factory invariant mismatch)
starbaser added 18 commits May 25, 2026 13:32
…an conn state

- package-mflows.py: delete body.metadata.user_id and
  body.diagnostics.previous_message_id keys outright (no zero-UUID
  placeholder). Replace client_conn and server_conn with sanitized
  Connection stubs so the wireguard config path and capture-time IPs
  don't ship in the bundled artifact.
- Re-derive src/ccproxy/templates/shapes/anthropic.mflow from a fresh
  capture using the corrected scrubber (4201 bytes).
- Delete src/ccproxy/templates/shapes/gemini.mflow — its tnetstring
  encoding was corrupted by the history rewrite step; re-capture is
  required (see CODEX_HANDOFF.md).
- Add CODEX_HANDOFF.md documenting session state, what's been done,
  and the remaining tasks: re-capture gemini, add provider-SDK e2e
  tests against the dev daemon, plus open follow-ups.
The package-mflows.py scrubber duplicated the existing apply-time
shaping system. The right approach: bundled .mflow is a faithful
capture; selective application happens at runtime via content_fields,
shape_hooks, merge_strategies, and strip_headers/preserve_headers.

Reverted:
- scripts/package-mflows.py — deleted
- .pre-commit-config.yaml — package-mflows-verify hook removed
- docs/fingerprint.md — 'Bundled vs personal shapes' section removed
- src/ccproxy/templates/shapes/anthropic.mflow — deleted; needs
  re-capture (filter-repo corrupted the original)

Kept (real apply-time fixes from this session):
- shape_capturer.py strip of x-ccproxy-flow-id at capture time
- EgressSanitizerAddon for x-ccproxy-* on outbound
- diagnostics added to anthropic content_fields
- HTTP_CONTENT_DECODING=0 in transport_kwargs
- _default_hooks() factory (ty diagnostic fix)

CODEX_HANDOFF.md rewritten with the corrected plan: re-capture both
bundled .mflow files, extend shaping config (content_fields,
shape_hooks) to cover per-user fields, add provider-SDK e2e tests
against the dev daemon. Bundled scrubbing as a separate packaging
step is explicitly rejected.
Previously load_hooks mutated singleton HookSpec objects without
resetting params, causing stale configuration to persist across repeated
loads. Now explicitly clears spec.params before validation to ensure
clean state.
Enables automated WSL2 validation by spinning up a disposable Windows 11
KVM VM, importing the .wsl artifact, running the PowerShell test harness
inside the guest, and collecting results via HTTP.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants